How statistical are quantum states? 
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A novel no-go theorem is presented which sets a bound upon the extent to which '\l/-epistemic ' 
interpretations of quantum theory are able to explain the overlap between non-orthogonal quantum 
states in terms of an experimenter's ignorance of an underlying state of reality. The theorem applies 
to any Hilbert space of dimension greater than two. In the limit of large Hilbert spaces, no more 
than half of the overlap between quantum states can be accounted for. Unlike other recent no-go 
theorems no additional assumptions, such as forms of locality, invasiveness, or non-contextuality, 
are required. The result continues to hold in the presence of a small but finite amount of noise, and 
is open to experimental verification in a sufficiently precise experimental arrangement. 
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One of the most remarkable features of a quantum sys- 
tem is that of non-orthogonality: it is possible to prepare 
two different pure quantum states which cannot be per- 
fectly discriminated by a single ideal measurement. No 
such property exists in classical systems, where two dif- 
ferent classical states can always by perfectly discrimi- 
nated by a suitably ideal measurement. 

Statistical interpretations of the quantum state (see [l|- 
0] and references therein) present a natural way to under- 
stand this feature of non-orthogonality. Suppose that a 
'quantum state' is only a statistical state, associated with 
a probability distribution over some underlying physical 
states. Even an ideal quantum preparation procedure 
cannot uniquely fix this underlying state of the system 
but instead prepares one of a number of possible such 
states, with a well defined probability. In such an inter- 
pretation, when two quantum states are non-orthogonal, 
the associated probability distributions will overlap: the 
preparation procedures each have a non-zero probability 
of preparing the same physical states, and this is why 
they cannot be perfectly distinguished. This has been 
called a ^-epistemic interpretation^, as the idea is 
that the wavefunction does not itself represent a physi- 
cal state, but instead represents an epistemic uncertainty 
about the physical state. 

A number of recent no-go theorems have explored 
the consequences of attempting to construct such 
cpistcmic interpretations of quantum theory. This paper 
contributes a new no-go theorem, demonstrating that any 
attempt to explain non-orthogonality entirely in terms of 
an epistemic overlap is impossible: there are no 'maxi- 
mally epistemic' interpretations of quantum theory. Us- 
ing an empirically testable measure, it is shown that in 
the limit of large Hilbert spaces, no more than half of 
the overlap between quantum states could be explained 
in terms of an overlap of probability distributions. Un- 
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like the other recent results, the theorem presented here 
makes no additional assumptions beyond the definition 
of a ^P-epistemic theory. This means that this theorem 
is the only such theorem to apply, for example, to the 
constructive ^-epistemic theory presented in [9j. 

Ontological models and ^ -epistemic theories.- Two 
principal assumptions are involved in analysing whether 
a theory is ^-epistemic : 

1. After a physical procedure prepares a quantum 
state, the system is actually in a physical state 
A. This physical state is not necessarily identified 
with the quantum state; 

2. The probabilities of getting the results of a mea- 
surement procedure on the system are wholly de- 
termined by the physical state A and the measure- 
ment procedure. The preparation procedure only 
influences the measurement outcomes indirectly, 
through the possible physical states prepared. 

These assumptions are developed into the ontological 
models formalism in[3|, H(il - [l3j . where the physical state 
A, is referred to as an ontic state. 

• A preparation procedure, which prepares the quan- 
tum state V)i "will prepare some ontic state A with 
probability /x^(A): 

/V( A )>° H\ 

• A measurement, M, has a number of possible out- 
comes {Q}, and a probability £m(Q\X) of obtaining 
a particular outcome Q, given the ontic state A: 



6f (Q|A) > o 
E Q 6/(Q|A) = i w 

• An ontic model will reproduce the results of quan- 
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turn theory 1 if, and only if: 

//v(A)MQ|A)dAH(Q |^)| 2 (3) 

Some immediate consequences of this formalism will be 
useful. 

The set = {A : /i</>(A) > 0} is the set of all pos- 
sible ontic states which may occur when preparing the 
quantum state \<p). 

If one of the outcomes of the measurement M 
is represented by the projection onto \<f>), then 
/ /v(A)£m (0|A)dA = |(0 \(f>)\ 2 = 1. It follows that 

VAeA 6/(0|A) = l (4) 

If \<p) is orthogonal to \<p) then / /x ¥ ,(A)^M(<^|A)dA = 
\(4> \ip)\ 2 = 0. It follows that 

VA G A v £ M (0|A)=O (5) 

which implies 2 A v n = . 

If the quantum two states and |0) are non- 
orthogonal, then: 

y^(A)fjif(0|A)dA = |(0 |^)| VO (6) 

This allows the possibility that A^, n A^, is not empty. 

A \E'-epistemic theory is defined^ to be one in which 
there exist at least two distinct non-orthogonal quantum 
states, \ip) and \<j>), for which A^ n A^ ^ 0. In such 
a theory, the fact that the quantum states \ip) and \<p>) 
cannot be discriminated by a single ideal measurement 
is, at least partially, explained by the fact that there is 
some A G A^ (~l A^. As all the results of any measurement 
M are determined by £m(<2|A), and both \ip) and \<j>) can 
prepare ontic states in (~l A^, it follows that there is 
no possible measurement outcome which can occur with 
zero probability for but with certainty for \<p) (and 
vice versa). 

The term J A ^ /!,/,( A)e?A gives a measure of the epistemic 
overlap between the quantum states \ip) and \<p). This is 
bound by the measure of the quantum state overlap. As 
VA G A , ( M (<p\X) = 1 

/ fj,^(X)d\ = / ^(A)6/(0|A)dA 

< J ^(A)6/(0|A)dA = |(0|^)| 2 (7) 

The degree of epistemic overlap, Q [<p, ip], between two 
states can now be defined 3 via: 

/ n^(\)d\ = rt [<f>,ip] \ (4> \ip)\ 2 (8) 



A maximally vE'-epistemic theory would have VL [cp, ip] = 
1 for all pairs of quantum states. The quantum state 
overlap \{<p \ip)\ 2 would be accounted for entirely in terms 
of the overlap in the ontic states A^ n A^ . 

By contrast, a maximally ^-ontic theory is one in 
which A,/, (~l A^ = for all pairs of distinct quantum 
states, and so fi [cp, ip] = . In such a theory, no two dis- 
tinct quantum state preparations \ip) and \<p) could pos- 
sibly produce the same ontic state A. For a maximally 
\l/-ontic theory, therefore, given the ontic state A it is al- 
ways possible to identify the quan tum state hi). The de 
Broglie-BohmfllOJ], Everett [T^ and CSL[r| interpre- 
tations of quantum theory are all examples of maximally 
vp-ontic theories. 

It will now be shown that, in Hilbert spaces of dimen- 
sion greater than two, there are no models of quantum 
theory for which ft [cp, ip] = 1 for all states: there are no 
maximally ^-epistemic interpretations. Making a sym- 
metrical assumption that Q [cp, ip] is a constant, fi, be- 
tween all pairs of distinct states it will be shown that 
& < for a 3 dimensional Hilbert space, falling to il < | 
in the limit of large Hilbert spaces. No more than 50% 
of the quantum state overlap can be accounted for by an 
epistemic overlap of probability distributions. 

Three dimensional Hilbert space.- Consider the follow- 
ing states 4 of a three dimensional Hilbert space: 

|o), |6>, \c) 

\P) = (\a) + \b) + |c» , \m) = ^ (|a> + \b) - |c» 
\a + ) = ^(\a) + \c)), |o_) = ^(|o)-|c» 
\b+) = j 5 (\b) + \c)), \b-) = ^(\b)-\c)) 

where a) , \b) , \c) form an orthonormal basis, with the 
three measurements: 

M l : |a+> <a+ 1 , |o_> <a_|,|6><6| 
M 2 : |& + ><6 + |,|6_><6_|,|a)(a| 
M 3 : \a)(a\,\b)(b\,\c)(c\ 

The results are shown in Table fl] From Mi it can be 
seen that 

VAeA a , £ Ml (6|A) = 

VAeA p , £ Ml (a_|A) = (9) 

VA G A m , £ Ml (a + \\) = 

For any given A, it must be that £m x (b\X) + £mi ( a - 1 A) + 
Cmi(o+|A) = 1, so this means: 

A a n A p n A m = (10) 

Similarly 

A c nA p nA m = (li) 



1 Additional structure is required to represent unitary operations, 
but these are not needed for this proof. 

2 Ignoring sets of measure zero. 

3 £7 [tp, ip] = 1 by definition. Where (<f> \ip) = 0, tt[<j>, ip] may take 
any value. 



4 These are also the states used in Clifton's proof of quantum 
contextuality 18] . For further connections to quantum contex- 
tuality, see ll9H . 
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TABLE I: Measurement outcomes 



A h n A p n A m 



(A Q U A h U A c ) n A p n A m = 



(12) 
(13) 



If there is any overlap in A p n A m then it must lie outside 
A a U A 6 U A c . 
Now M3 gives 



/ fi p (X)d\ 



n [a,p] 



(14) 



and similarly for |6) and |c). As At n A a = etc. 

^ (A)dA = (n[a,p] + n[fe, P ] + n[c,p]) 

'(A a UAbUA c ) 3 

It follows that: 



/ 

J (A 



jj,p(\)d\ < 



(n[a,p} + n[b,p} + n[c, P }) 
3 



(15) 
(16) 



but 



/ Li p (\)d\ = 
JA m 



il [m,p] 



(17) 



giving 

£1 [m,p] 



< 1 - 



(Q [a,p] +fi[6,p]+fi[c,p]) 



(18) 



If the ontic model has maximal epistemic overlaps with 
the \a) , \b) , |c) states, then fl [a,p] = Q[b,p] — f2 [c,p] = 
1, but this requires Q[m,p] = while |(m |p)| 2 = ^. 
This proves that a maximally ^-epistemic theory is not 
possible. 

Reducing the degree of epistemicness in the \a) , \ b) , |c) 
basis will allow some epistemic overlap between \p) and 
I to). For example, keeping Q[a,p] = fl[b,p] = 1 it is 
possible to reach il[m,p] — 1 with f2[c,p] = |. More 
symmetrically, Q[m,p] — 1 is possible with Q[a,p] = 

n[b,p] = n[c,p} = l. 

A basis independent measure of how epistemic a theory 
can be and still reproduce quantum statistics is given by 
letting [(f), tp] = fi be a constant for all non-orthogonal 
states. This gives: 



n < 



10 



(19) 



Higher dimensional Hilbert spaces.- The above results 
can be generalised to Hilbert spaces of dimension d > 3. 
In the limit of large d, O < |. 

Take the basis states 



\ ai ) ,\a 2 ) ,...\a d ) 



the superpositions 



k+) = 75 (k) + M) 

K> = 73 (E i= i, d -i k>) - 75 W 

and the measurements 

Mi : \ai) (ai \ , . . . , k_i) (a,_i | , 

k+i) ( a i+i I >• ■ •) |o<i-i) («d-i I 
k+) I j k-) I 
M d : \a 1 )(a 1 \,...,\ad)(a d \ 

Each measurement M,- now shows: 



A„ s n A Pd n A md = 1 
A ad n A Pd n A md = 



(20) 



leading to 

(u l A Qi )nA Pd nA md = 

From the M d measurement: 

/ v Pd ( x ) dx =y2^[ai,Pd\ \(ai \p d )\ 2 = \y\ 



(21) 



^ [di,Pd] 
(22) 
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jf /x P(J (A)dA < 1- i^f2[oi,p d ] 



giving 



(l-§) < 1- -^y^^[aj,p d ] 



(23) 



(24) 



Once again, ^[a^p^] = 1 would require Q[md,Pd] = 0, 

even though \{nid \pd)\ 2 = (l — |) — » 1. For a theory to 
be maximally 'J'-epistemic in a specific basis, then as d — > 
oo it must become maximally 'I'-ontic between at least 
some states arbitrarily close to each other. To obtain 
a full overlap [nid , Pd] — 1 by symmetrically reducing 
the overlap with the |oi) basis would require fl [cii,pd] = 

.4(d_i)->o. ; 

The basis independent measure of how epistemic this 
allows the theory overall sets fi [<f>, ip] — Sl(d) as before 
giving 



2d 2 - Ad + 4 



(25) 



In the limit d gets large, £l(d) < \. 

Experimental Errors. - The ideal measurement depends 
upon obtaining zero probability of a set of outcomes. In 
any real experiment, there will be some noise, giving a 
value e > 0. We follow [j| in introducing a measure 
of overlap in the ontic state distribution which can be 
compared to experimental data in the presence of noise: 



[^{\),^{\)]d\ = Lo[<f),i)]\{<f)\iP)\ 2 (26) 



ui [(f), ijj) replaces fi [(f), tp] as the measure of the degree 
to which the quantum state overlap is explained by the 
overlap in probability distributions. It is shown in the 
appendix that when the errors are of order e, the bound 
for a 3 dimensional Hilbcrt space becomes 

u[™,P\ < x _ (^ [a,p\+u)[b,p]+u[c,p]) + 



For a constant ui, 
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(28) 



requiring e < to rule out maximally ^-epistemic the- 
ories. 

For Hilbcrt spaces of dimension d, 



w [pd,rn d ] 1 



< l-\^io[p dl a t ] + ]-d 2 {d+l) 



For a constant tu(d), 

io(d) < 



d 2 + \d 4 {d+l)e 
2d 2 - id + 4 



(29) 
(30) 



requiring 

2(d-2) 2 
£< d 4 {d+l) 

In the limit of large d, this gives 



w(d) < g + e X 



(31) 



(32) 



requiring e < 

Although this requires a very high precision experi- 
mental test, sufficiently small finite e will nevertheless en- 
sure that w{d) < 1 and in large Hilbert spaces w(d) < \. 
The theorem is therefore robust against finite precision 
loopholes. 

Conclusions.- There exist constructive examples of ty- 
epistcmic models, which can reproduce all the statis- 
tics of quantum theory (such as the LJBR modcl|9]). 
It is therefore not the intention of this article to at- 
tempt to construct no-go theorems which aim to show 
that ^-epistemic theories are impossible simpliciter. Ex- 
isting no-go theorems have effectively sought to prove 
51 [(f), tp] — 0, but to do so must assume additional proper- 
ties, such as forms of locality |8|, factorisability of product 
states Q or non-invasiveness[7j . This should not down- 
play the significance of these theorems: any attempt 
to construct ^-epistemic theories must necessarily re- 
ject these properties, in much the same way that Bell's 
theorem[20j requires realist interpretations of quantum 
theory to reject local causality. They help to map out the 
space of possible theories which can account for quantum 
phenomena. 

The no-go theorem presented here contributes to this 
task. It does not make additional assumptions so ap- 
plies even to theories that reject all of these additional 
assumptions, such as the LJBR model. The LJBR model 
demonstrates that f2 [(f), ip] > is indeed possible. This 
no-go theorem shows that there are empirically verifi- 
able bounds upon how large f2 [<f>, if)] can become, even if 
all other assumptions are dropped. In the limit of large 
Hilbert spaces, no more than half the quantum state over- 
lap can be accounted for in terms of epistemic overlaps. 
It is not clear that the LJBR model reaches the boundary 
derived in this paper. This would leave open two possi- 
bilities: either a more epistemic model than LJBR can 
be constructed, or else a tighter bound may be derived. 

The question of whether the quantum state is real or 
statistical has been a matter of debate since the early 
days of quantum theory. It would be amusing if the an- 
swer turned out to be: 'about half and half. 
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Error free proof.— The error free proof involves de- 
composing the overlap between the \p) and \m) states: 

p n M = 

(PnMnfiuBu C)) u (p n M n ^{A u B u C)) 
= (PnMn(4uBuC)) 
u ((P n ubu C)) n (M nn(AuBu C))) 

(33) 

In the ideal case, for any measure v [X] , we have 
v [P n M n (A U B U C)] = and 

i/ [P n M] < min (i/[P]-i/[Pn(AUflU C)] , (34) 
u[M]-v[Mn(AUBUC)]) (35) 

As 

v [P n (A u b u c)] = i/ [P n A] + 1/ [P n P] + i/ [P n C] 
-i/ [P n A n P] - v [P n P n C] - u [P n A n C] 

+f [FninfinC] 

(36) 

and z/ [P n A n B] =0 (and similar combinations) then 

i/[?n M] < 

min (v[P] — v[P n A] — v[P n B] — u[P nC\ , 
v [M] — v[MC\A] — u [M nB]-v [M n C]) 

(37) 

Using the measure \i [P D M] = f2 [p, m] | (p |m) | etc., we 
recover the ideal result. 

Noise tolerance.— This proof does not appear noise 
tolerant, as it requires some measurement outcomes to 
have a probability of zero. We now show how to gen- 
eralise the result to situations where the error in such 
measurements is e > 0. 

In general, for any measure v [X] : 

v [P n M n (A U B U CO] = 

v [P n M n A] + v [P n M n b] + v [p n m n c] 
-v [P n m n A n p] - ^ [p n m n p n C] 
-v [p n m n A n <7 j + v [p n M n A n P n C) 

(38) 

and 

i/[PnMnAnfl] + i/[PnMnflnC] 

+f[PnM nine] >u[PnMnAr\BnC] 



The worst case scenario is 

v [P n m n (A u p u CO] < f [P n M n A] 



+z/ [P n M n P] + v [P n M n C] 



(39) 



(40) 



For 



v [P n -(A u p u C)] = f [P] - f [P n A] - j/ [P n P] 
-i/[PnC]+v[PninB]+!/[PnBnC] 
+i/[Pnin(7]-i/[Pn4nBnC] 

(41) 

the worst case scenario is 

v[Pn ->(A uBuC)] = u[P]-u[PnA]-v[Pn b] 
-v[Pr\C] + v[Pr\Ar\B] + v[Pc\Br\C] 

+v[PC\AC\C] 

(42) 



giving an expression suitable for non-zero overlaps: 

v [P n M] < v [P n M n A] + v [p n M n P] + 1/ [p n M n C] 
+ min [(i/ [P]-i/[PnA]-i/[PnB]-f[PnC] 

+ i/[Pn J 4nP] + i/[PnBnC] + j/[PninC]), 
(i/ [M] - u[M n A] - u [M n P] - v [M n C] 

[M n A n P] + 1/ [M nPnC] + z/[M nAn C])] 

(43) 

Measure of epistemic overlap. We now need a 
measure which can be compared to experimental data. 
We follow [5] in using: 



^[$n*] = J min [^(A),jty(A)]dA 



(44) 



It can be straightforwardly verified that this generalises 
using: 

t/[#J = / fi ( f ti (X)dX = 1 
i/[rii$i] =/min i [p 0i (A)]dA 

i/[Ui«i] =/max i [p0 j (A)]dA (45) 
z/ [$ n -■*] = / (^(A) - min [^(A), fi^(X)}) dX 

= v [$] - z/ [$ n *] 

to include all the required measure theoretic relations 
such as z^ [A UP] = z/ L4] + z/ [P] - y[inB] and 
v[An(Bl)C)] = u [(A n P) U (A n C)] etc. 

Degree of epistemic overlap.— For any £m(<9|A) 



min[p0(A),^(A)]^M(Q|A)dA < / /^(A)^/ (Q\X)dX 

(46) 

nun [//..(A).//, (A)] £,.w (<7|A u/A '_ / /i^,(A)^ M (Q|A)dA 

(47) 



and 



so 



/ min[/i0(A),p^(A)]^M(<9|A)dA 

< min [J w(X)£ M (Q\X)dX,J ^{X)^ M (Q\X)dX] 

(48) 

As J2q€m(Q\X) = 1, 

1/ [3 n #] = Eq / min [ M0 (A), M0 (A)] ^A/(Cj|A)dA 

< E Q min [J ^(X^MiQWdXj ^(X)£ M (Q\X)dX] 

(49) 

This will be true for any measurement procedure M. 

If the M measurement is a projection onto \tp) and its 
orthogonal subspace, 



mil 



^(X)^ M {Q\X)dX, / ^{X)£ M (Q\X)dX 



= 1(0 i^)r 

(50) 



hence z^ [$ n ^] < | ((f) \ip) \ ■ It can easily be verified that 
v [$ n *] = if and only if A^nA^, = 0, and v [$ n *] = 1 
if and only if \x^(X) = fX^(X). 

We can now define a degree of epistemic overlap u> [(f), ip] 
that can be empirically bounded even in the presence of 
noise: 



z4$n*] =u[<f>,ti>]\(<i> |V)r 



(51) 
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Comparison to experimental data. To compare 
to experimental data, if /m(Q|0) is the measured fre- 
quency with which an outcome Q occurs, given the quan- 
tum state is \cj>), then 

i/[$n*] <^min[/ M (W),/M(Q|V)] (52) 
Q 

For the intersections between more than one quantum 
state, this generalises to 

^ Z)Q mm i [fM{Q\4>i)\ 

where the terms on the right hand side can be read off 
directly from the measurement results. 

Again, we find v [0^$;] = if and only if C\iK < f >i = 0. 
However, when there is noise in the measurement, each 
Q outcome will contribute e in place of a zero, so in a 
e?-dimensional Hilbert space, v [ni$i] < de. 

Noise tolerant proof.— For a three dimensional 
Hilbert space, this gives: 

u>[m,p] \{m \p)\ 2 < 1 -ui[a,p] \(a \p)\ 2 -w[b,p] \(b \p)\ 2 

-u [c,p]\(c |p>| 2 + 18e 
(54) 



and so 



^ [m,p] < 1 _ co [a,p]+u)[b,p] +uj[c,p] ^ 



For a constant lj, 
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(55) 



(56) 



Generalising to d dimensional Hilbert spaces is 
straightforward. 



The worst case scenario is 

v[PC\ M n (UiAi)} <^v[P^Mr\Ai] (58) 



and 



v [P n -^{UiAi)} < v [P] - £. 1/ [P n At] 



(59) 



giving 



v [P n M] < J2i v [ p n M n A t ] 

+ min [(v [P] -Ei^pniJ + i . ,[ P nAn A,-]) 
(i/ [M] - £. * [M ni t ] + | E,^- " [M n 4 n A,})' 

(60) 

Each of the d terms i/[FnMnii] and the \d(d - 
1) terms (/[Fn^n Aj] (or f [M n Ai H A,-]) contribute 
terms of order cfe, giving 

\m d )\ 2 < \p d ,ai}\{p d \ai)\ 2 +^-d 2 (d+i)e 



(61) 



so 



/ 2\ 2 1 1 

[Pd,m d ] <l--Y,"[Pd,a l } + -d 2 (d+l)e 



d 

For a constant to(d), 

ic(d) < 



d 2 + ±d 4 (d+l)e 
2d 2 -Ad + 4 



(62) 



(63) 



p n M = (P n M n (u,A0) u (p n M n ^(u.A,)) (57) 
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